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Executive Summary 


The evaluation of the Texas Rural Technology (R-Tech) Pilot sought to understand how districts 
implemented R-Tech grants, the effects of implementation on student and teacher outcomes, as well as 
the cost effectiveness and sustainability of R-Tech. The Texas legislature (80th Texas Legislature, 
Regular Session, 2007) authorized the creation of R-Tech in order to support the state’s small, rural 
districts in implementing technology-based supplemental education programs. R-Tech grants were 
intended to support supplemental educational programs, including online courses, offered outside of 
students’ regularly scheduled classes (e.g., before or after school). Districts that received grants were 
required to provide students in Grades 6 through 12 with access to technology-based instructional 
resources for a minimum of 10 hours a week; however, the grant did not establish minimum requirements 
for students’ use of R-Tech resources. 

In establishing R-Tech, the legislature required that the program be evaluated to assess its effects on 
student and teacher outcomes, as well as the program’s cost effectiveness. In addressing these goals, the 
evaluation considered the following research questions: 

1 . How is R-Tech implemented across grantee districts and schools? 

2. What is the level of student participation in R-Tech? 

3. What is the effect of R-Tech on teachers? 

4. What is the effect of R-Tech on student outcomes? 

5. How cost effective is R-Tech? 

The evaluation is made up of two interim reports (December 2008 and February 2010) and a final report 
(fall 2010). The findings presented here are drawn from the evaluation’s final report (fall 2010). The 
report considers outcomes for 63 districts that received Cycle 1 grant awards 1 across the 2-year grant 
period (May 2008 through May 2010). 

KEY FINDINGS BY RESEARCH QUESTION 

The sections that follow present key findings relative to each of the evaluation’s research questions. 
Results are drawn from data collected across the full 2-year implementation period for Cycle 1 districts. 

Research Question 1: How Is R-Tech Implemented Across Grantee Districts and 
Schools? 

The following sections present information about the types of programs districts implemented using 
R-Tech funds and finds that some districts encountered challenges in implementing supplemental 
programs that caused them to revise their plans. District representatives explained that many students 
resisted participating in programs offered before or after school. Further, some students were not able to 
participate in R-Tech services because of conflicts with extra-curricular activities and bus schedules that 
limited their ability to arrive early or stay after school. 


'To date, there have been three cycles of R-Tech grant awards. 
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• Most districts (87%) 2 implemented R-Tech as a self-paced program focused on tutoring, remediation, or 
credit recovery. Self-paced programs provided access to online lessons that students worked through at 
their own pace. Many self-paced programs included diagnostic assessments of students’ individual 
learning needs and tailored instruction based on assessment outcomes. Some programs enabled students 
to complete entire courses online, allowing students to make up credit for incomplete or failed courses. 

• About 30% of Cycle 1 districts offered dual credit coursework using R-Tech funding. Dual credit 
courses enable students in Grades 1 1 and 12 to take courses that fulfill high school graduation 
requirements and earn college credit. R-Tech districts implementing dual credit courses partnered with 
community colleges and universities to provide instruction, and some programs were facilitated by 
regional Education Service Centers (ESCs). 

• Although R-Tech was intended to support districts’ efforts in implementing supplemental educational 
programs offered outside the regularly scheduled school day, a substantial proportion of Cycle 1 
districts (40%) implemented R-Tech as part of classroom instruction (i.e., non-supplemental programs). 
Many districts used R-Tech funding to update their computer labs and teachers scheduled class time in 
the lab for students to access resources. Two districts implemented R-Tech as a technology immersion 
program and used funding to support the purchase of laptop computers for all teachers and students in 
Grades 6 through 12. Students and teachers used laptops throughout the school day and schools may 
have permitted students to take laptops home. 

Research Question 2: What Is the Level of Student Participation in R-Tech? 

The sections that follow summarize student participation in R-Tech across the 2-year grant period and 

discuss how most students were identified for R-Tech services. 

• In the second year of implementation, a larger proportion of districts and campuses reported greater 
numbers of students participating in R-Tech. About 1,400 students participated in R-Tech in summer 
2008, about 8,800 participated in fall 2008, and nearly 12,800 participated in spring 2009; while 
approximately 3,300 students participated in summer 2009, about 13,000 participated in fall 2009, and 
nearly 14,000 participated in spring 2010. 

• The average amount of time each student accessed R-Tech services during the grant’s second year 
decreased from levels reported in year 1, with approximately half of all students accessing R-Tech 
services less than 2 hours a week. Districts varied in how they implemented R-Tech and how students 
were identified to receive services, but most campuses identified students because of weak academic 
outcomes, including poor Texas Assessment of Knowledge and Skills (TAKS) scores, failing grades, 
and teacher referrals. 

• Student resistance, transportation challenges, and scheduling conflicts created the greatest barriers to 
participation in supplemental R-Tech programs. Districts addressed participation barriers by expanding 
available times and locations for R-Tech services, requiring participation of some students, and 
increasingly integrating R-Tech services into regular classroom instruction (i.e. non-supplemental 
implementation) . 


2 The percentage of districts included in each program type will not total to 100 because districts were able to 
implement more than one type of program. Districts were able to implement separate programs in their middle and 
high schools. For example, a district may have implemented dual credit instruction in its high school, but offered a 
self-paced tutoring program in its middle school. 



Research Question 3: What Is the Effect of R-Tech on Teachers? 


In grant applications, all Cycle 1 districts indicated that R-Tech resources would be used to expand 
teachers’ access to technology-based professional development activities; however, results from teacher 
surveys and focus group discussions suggest that many teachers were unaware of the R-Tech resources 
available to them and that few teachers participated in R-Tech professional development opportunities 
across implementation years. However, teachers working in districts in which R-Tech was incorporated as 
part of regular instruction reported using resources to a greater extent than teachers in districts 
implementing supplementary programs. Teachers in non-supplementary districts reported using R-Tech 
resources to differentiate instruction, provide remediation and support for struggling learners, and to 
reinforce concepts taught in class. 

• According to results from surveys and focus group discussions, teachers on R-Tech campuses received 
limited training. Specifically, less than 5% of teachers responding to the fall 2008 survey (54 
individuals), 38% of spring 2009 survey respondents (215 individuals), and 29% of spring 2010 
respondents (392 individuals) knew they had participated in R-Tech professional development. Most 
teachers receiving R-Tech Paining attended sessions provided by vendors onsite and in-person which 
addressed preparation for standardized tests, integrating instructional technology into classroom 
instruction, and working with at-risk students. 

• Beyond professional development opportunities, teachers reported that they benefitted from the 
increased access to technology provided by R-Tech, which allowed them to enhance their lesson plans, 
provide visual and auditory examples of lesson content, differentiate instruction, and provide 
remediation to struggling students. Additionally, teachers reported increased student engagement when 
students used instructional technology resources. 

Research Question 4: What Is the Effect of R-Tech on Student Outcomes? 

The sections that follow present results from analyses of R-Tech on students’ TAKS and attendance 
outcomes. Given differences in the availability of data, the evaluation’s analyses of attendance outcomes 
are limited to R-Tech’ s first implementation year, while analyses of TAKS outcomes consider the 
program’s full 2-year implementation period. 

• Comparisons of changes in the percentages of R-Tech participants and non-participants 3 who met 
TAKS passing standards from 2008 (the year prior to R-Tech implementation) to 2010 (the grant’s final 
year) indicate that R-Tech participants had larger gains in TAKS passing rates than non-participants in 
mathematics and science. These gains were not found in reading/English language arts (ELA) or social 
studies. 

• Students who spent more time participating in R-Tech services did not experience improved testing or 
attendance outcomes relative to students who spent less time in R-Tech. However, researchers were not 
able to control for unobserved student differences that may have affected outcomes. For example, 
students who spent more time in R-Tech may have been at greater academic risk, requiring more 
remediation time than students who spent less time using resources. 

• Students participating in self-paced programs experienced reduced TAKS scores in reading/ELA and 
mathematics relative to R-Tech students who participated in other program types; however, self-paced 
programs had no demonstrated relationship to TAKS outcomes in science and social studies. R-Tech 
dual credit and distance learning programs did not demonstrate a statistically significant relationship 


3 Non-participants are students who attended R-Tech campuses but did not receive R-Tech services. 
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with students’ TAKS reading/ELA and mathematics scores. Again, results should be interpreted with 
caution because it was not possible to control for the student characteristics that may have caused 
students to be identified for self-paced programs. If students identified for self-paced programs had 
more serious academic deficiencies than students identified for other types of R-Tech programs, then 
results may have been produced by unobserved student characteristics rather than program 
participation. 

• Students who received R-Tech services as non-supplemental instruction offered during the regular 
school day experienced improved TAKS testing outcomes in reading/ELA and mathematics relative to 
students who participated in supplemental R-Tech programs. Students who participated in R-Tech 
during the regular school day also experienced improved attendance outcomes. These findings suggest 
that R-Tech services implemented as part of regular instruction may improve students’ TAKS and 
attendance outcomes; however, the characteristics of students identified for supplemental services may 
have affected outcomes. That is, students identified for supplemental services may have struggled 
academically, while students participated in non-supplemental services irrespective of academic need, 
which may indicate that testing outcomes resulted from students’ academic characteristics rather than 
program participation. 

Research Question 5: How Cost Effective Is R-Tech? 

Similar to findings for R-Tech’ s effects on student achievement, readers are asked to use caution when 

interpreting the results of the evaluation’s cost effectiveness analysis. At the time of the report’s writing, 

only 31 Cycle 1 districts had accessed their full grant awards, and the remaining 32 districts had used only 

7 1 % of their grant funding. 

• In spite of substantial start up costs in terms of investments in technology resources, districts that 
implemented R-Tech for larger numbers of students experienced the lowest per-student program costs. 
Across Cycle 1 districts, the average per-student cost of providing R-Tech services in terms of state- 
provided grant funding across the 2-year grant period was $294. Districts that implemented programs 
serving 1,000 or more students experienced average per-student costs of $141, while districts that 
served fewer than 100 students had average per-student costs of $774. 

• Districts that implemented R-Tech as part of regular classroom instruction (i.e., non-supplemental 
programs) experienced substantially lower per-student costs than supplemental programs ($212 vs. 
$353, on average). The difference in costs results from differences in the numbers of students served. 
Districts implementing supplemental programs served an average of 346 students across the 2-year 
grant period, while districts implementing non-supplemental programs served an average of 692 
students. 

• More than half (58%) of principals responding to the spring 2010 survey reported that insufficient 
financial resources created a moderate or substantial barrier to continuing R-Tech after grant funds 
expired in May 2010. Most surveyed principals (60%) indicated that they would seek additional 
funding to continue the program, and 3 1 % indicated they would continue services by incorporating 
R-Tech into regular classroom instruction. 
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CONCLUSION 


The overarching finding of the evaluation is that rural districts struggled to implement supplementary 
R-Tech programs in which instruction was offered outside of the regular school day. Many rural students 
travel great distances to school and depend on buses for transportation. In many districts, bus schedules 
did not permit students to arrive early or remain after school in order to receive supplementary 
instruction. Conflicts with extracurricular activities, student work schedules, and family responsibilities 
also limited some students’ ability to participate in R-Tech programs, and some students simply refused 
to participate in instruction offered outside of the school day. 

Findings from the evaluation’s second interim report indicated that many districts revised their 
implementation plans in order to overcome these challenges. As a means to ensure greater student 
participation in R-Tech, many districts included services as part of the school day and encouraged 
teachers to use resources as part of classroom instruction. Findings from the 2-year evaluation indicate 
that districts that incorporated R-Tech as part of regular instruction (i.e., non-supplementary programs) 
experienced benefits relative to districts that adhered to the grant’s intent and implemented supplementary 
programs. The evaluation’s results indicate that districts implementing non-supplementary programs: 

• Served more students using R-Tech resources, 

• Experienced lower average per-student implementation costs, 

• Had better student outcomes in reading/ ELA and mathematics, 

• Improved attendance outcomes, and 

• Achieved greater teacher buy-in and support for grant goals. 

Recognizing the challenges that rural districts experience in implementing supplemental instructional 
programs and the benefits of including technology-based resources as part of classroom instruction, the 
evaluation recommends grant guidelines be revised to enable districts to include R-Tech services as part 
of regular instruction in addition to offering supplementary programs. Doing so will enable more students 
to access R-Tech resources, increase teacher awareness of services, reduce program costs, and may lead 
to improved achievement outcomes. 


For additional detail and discussion, the complete report is located at the following website: 
http://www.tea.state.tx.us/index4.aspx?id=2926&menu_id=949. 
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